In display systems, the panel is driven by drivers (column drivers and row drivers), which are controlled by display controllers. The primary function of the display controllers (TCON) is for timing control. A TCON might also integrate drivers to directly drive a panel. On the input side, the display controller is interfacing with host systems such as a graphic processing unit (GPU), an application processor with GPU integrated, or other type of processors like TV SoCs. In some situations, the display controllers are also integrated in SoCs. Commonly used interfaces between host systems and display devices are LVDS, DisplayPort, MIPI, V-by-one.
The display devices are generally programmed to work at a fixed timing such as 60 Hz or 120 Hz, which is the refresh rate measured as the number of frames per second (fps). The display is drawing the frame at this fixed refresh rate regardless of whether the frame is static or not. A self-refresh mode is a special TCON work mode where the display devices keeps drawing a previously captured frame in the TCON independently from the host system so that the GPU and other display related blocks in the host systems can be shut off for power saving. The self-fresh mode requires a frame buffer inside the TCON.
The self-fresh mode only addresses the case where the frame is completely static. When there is a slight change, even just a small motion for one pixel, the self-refresh mode needs to be switched off and the TCON needs to switch back to default work mode to take 60 fps data from the host system. In this case, the host system needs to read pixel data from the frame buffer at a fixed 60 frames per second even GPU or HD Video codec might be writing to the buffers at a much lower frame rate, or in some other situations writing the same repeating pixels between frames when there is a small change between frames. Related technology is the US Patent Publication US2010/0123727 A1 published on May 20, 2010 filed Nov. 18, 2008. This patent describes an invention that a module monitors the graphics activity and then configures the display device to operate at self-refresh mode.
The strong temporal correlation between successive frames can be exploited to re-define the data as key frames plus vector data with low motion frames being skipped. The vector data (motion vectors for video cases) can be calculated by CPU or GPU, calculated off-line in the cloud by servers, or beforehand in a personal computer. With the new vector data, the host system can operate at desired frame rate, for example 24 fps for film or even lower frame rate such as 15 fps for low motion animation drawing. The display device (TCON) will receive fewer frames with additional vector data that is frame, block or pixel based. The TCON will use the 2-dimensional (2D, horizontal plus vertical) vector data plus the pixel data to generate the interpolated frames (the skipped frames). These 2D motion vectors are not the same as the motion vectors used in digital video codecs. Those motion vectors are optimized for coding efficiency and in general are not equal to motion vectors that describe the true motion in the image.
Frame based vectors describe a model that can be used to determine block or pixel based motion vectors. The block or pixel based motion vectors can be determined using the app processor CPU/GPU or in the TCON itself. An example of a using key frames and a frame based model for motion vectors is when implementing a special effect for a user interface, such as rotating the screen image to match a tablet's orientation. During the special effect, the “motion” is already known and the GPU is using the frame based model to do the warping. For smaller amounts of warping, the frame interpolation algorithm in the TCON is capable of providing a similar effect, but at a potentially higher frame rate and without impacting the bandwidth of the application processor, resulting in a more responsive system.
When all vectors are force to zero, most blocks in host system can be shut off for power saving and the TCON falls back to the self-refresh mode. When vectors are not zeros, the host systems can operate at lower frequency, rendering the pixels at lower frame rate, which also saves power for active state. When contents' native frame rate is lower than 60 fps, the frame interpolator in TCON helps improve the picture quality by creating smooth motion and reducing motion judder.